A control-theoretic perspective on optimal high-order optimization

Abstract We provide a control-theoretic perspective on optimal tensor algorithms for minimizing convex function in finite-dimensional Euclidean space. Given $$\varPhi : {\mathbb {R}}^d \rightarrow {R}}$$ ? : R d ? that is and twice continuously differentiable, we study closed-loop control system governed by the operators $$\nabla \varPhi $$ ? ^2 2 together with feedback law $$\lambda (\cdot )$$ ? ( · ) satisfying algebraic equation $$(\lambda (t))^p\Vert \nabla (x(t))\Vert ^{p-1} = \theta t p ? x - 1 = ? some $$\theta \in (0, 1)$$ ? 0 , . Our first contribution to prove existence uniqueness of local solution this via Banach fixed-point theorem. present simple yet nontrivial Lyapunov allows us establish global under certain regularity conditions analyze convergence properties trajectories. The rate $$O(1/t^{(3p+1)/2})$$ O / 3 + terms objective gap $$O(1/t^{3p})$$ squared gradient norm. second two algorithmic frameworks obtained from discretization our continuous-time system, one which generalizes large-step A-HPE framework Monteiro Svaiter (SIAM J Optim 23(2):1092–1125, 2013) other leads new p -th order algorithm. While discrete-time analysis can be seen as simplification generalization (2013), it largely motivated aforementioned analysis, demonstrating fundamental role plays acceleration clear advantage brings design. A highlight show all discuss minimize norm at $$O(k^{-3p})$$ k , complements recent Gasnikov et al. (in: COLT, PMLR, pp 1374–1391, 2019), Jiang 1799–1801, 2019) Bubeck 492–507, 2019).